Skip to content

perf(dataset): skip unnecessary completed-dataset decode — built-in prefilter fast path#89

Merged
Lythaeon merged 1 commit into
mainfrom
perf/completed-dataset-built-in-skip
Apr 9, 2026
Merged

perf(dataset): skip unnecessary completed-dataset decode — built-in prefilter fast path#89
Lythaeon merged 1 commit into
mainfrom
perf/completed-dataset-built-in-skip

Conversation

@Lythaeon

@Lythaeon Lythaeon commented Apr 9, 2026

Copy link
Copy Markdown
Owner

Description

Reduce completed-dataset processing cost without changing SOF API contracts or feature scope.
This PR teaches the dataset path to classify transactions from sanitized transaction views using the existing compiled prefilter mechanism, so SOF decodes owned transactions only when dispatch actually needs them.

Changes

Detailed list of what changed:

  • crates/sof-observer/src/app/runtime/dataset/process.rs: add completed-dataset view extraction and prefilter-driven dispatch flow that can avoid owned transaction decode when compiled prefilters prove a transaction is irrelevant.
  • crates/sof-observer/src/framework/host/core.rs: add view-based host-side classification for compiled transaction prefilters and report when full decoded fallback is still required.
  • crates/sof-observer/src/framework/plugin.rs: keep the fast path on the existing TransactionPrefilter contract rather than introducing a new plugin API.
  • crates/sof-observer/src/framework/host/tests.rs: extend coverage for view-vs-decoded classifier equivalence and fallback requirements.
  • Any architecture/runtime/infra implications: the optimization is built in and automatic when the host can classify from compiled prefilters; it is not an opt-in plugin hook.

For slice-related changes, include:

  • Affected slices: dataset reconstruction/processing, framework host classification, plugin prefilter hot path.
  • Cross-slice communication changes (if any) and why: dataset processing now asks the framework host to classify from sanitized views before deciding whether full decode is needed.
  • Migration requirements (if any): none; existing plugin API contracts remain intact.

Motivation

Business motivation:

  • Improve throughput and reduce wasted compute while keeping the current feature set stable.

Technical motivation:

  • Completed datasets already have enough serialized structure for SOF to run compiled prefilters before full owned decode.
  • Plugins that still require decoded classification continue to get the old fallback path.

Alternative approaches considered:

  • Introducing a new opt-in plugin hook for view classification.
  • Leaving the decode optimization as a manual/plugin-specific path.

Scope and impact

  • Affected slices: dataset/runtime hot path and framework classification.
  • Data/API changes: none.
  • Backward compatibility: fully backward compatible.
  • Performance impact: positive on completed-dataset paths when compiled prefilters can reject or isolate transactions before owned decode.
  • Security impact: none.

Testing

  • Unit tests
  • Integration tests
  • Manual verification
  • Performance checks (if applicable)
  • Security checks (if applicable)

Commands/results:

cargo fmt --all
cargo test -p sof --lib --no-run
cargo test -p sof --lib transaction_prefilter_view_classification_matches_decoded -- --nocapture
cargo test -p sof --lib transaction_view_prefilter_marks_manual_plugins_for_full_decode -- --nocapture
cargo test -p sof --lib completed_dataset_prefilter_decodes_only_matching_transactions -- --nocapture

Results: passed.

Performance checks:

  • Local fixture A/B from the earlier prototype showed a completed-dataset loop improvement from roughly 669 ms to 470 ms over 5000 iterations.
  • This PR keeps the same optimization direction but moves it onto the built-in prefilter contract instead of a new opt-in hook.

Related issues and documentation

  • Fixes:
  • Related: packet/FEC and completed-dataset perf roadmap follow-up.
  • Architecture docs: docs/architecture/README.md
  • Relevant ARD/ADR:
  • Operations/runbook updates: none.

Reviewer checklist

  • Code follows project standards and architecture constraints
  • Slice boundaries are respected (docs/architecture/ard/0003-slice-dependency-contracts.md)
  • Tests added/updated and passing
  • Documentation updated (README/docs/operations as needed)
  • No undocumented breaking change
  • Performance trade-offs documented where relevant
  • Security considerations addressed where relevant

Additional notes

This PR is the top of the current stacked chain and should merge after refactor/import-hygiene.

@Lythaeon Lythaeon force-pushed the refactor/import-hygiene branch from 02043f4 to 6efd16c Compare April 9, 2026 13:18
@Lythaeon Lythaeon force-pushed the perf/completed-dataset-built-in-skip branch from 232c4bb to d890efe Compare April 9, 2026 13:20
@Lythaeon Lythaeon changed the base branch from refactor/import-hygiene to main April 9, 2026 13:20
@Lythaeon Lythaeon merged commit 9ec7bd4 into main Apr 9, 2026
2 of 4 checks passed
@Lythaeon Lythaeon deleted the perf/completed-dataset-built-in-skip branch April 9, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant